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(57) A monitor which can detect respiration of a 
sleeping person without being affected by the attitude 
of the sleeping person orthe indoorillumination light and 
can easily evaluate detected respiration quantitatively 
through image measurement. The monitor comprises 
means (1) for projecting a specified illumination pattern, 
means (5) for picking up the image of projected light con- 
tinuously, means (7) for calculating inter-frame moving 
amount of the illumination pattern from the image of two 
frames acquired by the image pickup means at different 
times, means (8) for generating a moving amount wave- 
form data comprising inter-frame moving amounts ar- 
ranged in time series, and means (9) for detecting the 
movement of an object from the moving amount wave- 
form data. 
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Description 

Technical Field 

s [0001] The invention of this application relates to a monitor. More particularly, the present invention relates to a 
monitor capable of monitoring the body movement etc., of a sleeping person in a noncontact manner with image meas- 
urement. 

Background Art 

[0002] With the advent of an aging welfare society, the importance of home health monitoring for disease prevention 
and self care is expected to continue to grow, and various bioinstrumentation methods have been proposed. An aged 
but healthy normal person who can live an ordinary life can be mentioned as an example of a person to be the subject 
of such home health monitoring. However, under the present circumstances, such a person lacks the motivation to 

is exercise health control and perform a troublesome device operation, because he/she does not need imminent health 
care. Therefore, a proposal has been made of various bioinstrumentation devices capable of measuring biomedical 
information under an unconscious condition without any device operation. As measurement methods employed in such 
bioinstrumentation devices, there are known a method of arranging temperature sensors in a bed or a bottom mattress 
and, from a temperature distribution, recording body movements at the time of getting into bed, at the time of rising 

20 from bed, and during sleep in bed, a method of deriving an electrocardiogram from an electrically conductive sheet 
and a pillow cover, and a method of measuring respiration and heart beat during sleep by use of a load sensor such 
as a highly accurate strain gauge or a load cell. 

[0003] Recently, a proposal has been made to use such bioinstrumentation technology as a security device applied 
in the safety confirmation of an aged person besides the usage of health monitoring. For example, a leaving-bed sensor 

25 has been generally sold for detecting the leaving or falling out of bed of a person during sleep by measuring an elec- 
trostatic capacity etc. Additionally, an apnea detector is well known in which an apnea state , caused by an apnea 
syndrome by which breathing ceases during sleep , is detected by measuring respiration with a pressure-sensitive 
sensor, and a third party can be informed of the state (e.g., Japanese publication of unexamined patent application 
No. (hereinafter referred to as JP-A-) 2000-107154). 

so [0004] As a matter of course, respiration provides a very useful clue for immediately knowing the health condition of 
a person. Respiration measurement performed during sleep is expected to be useful not only in detecting an apnea 
syndrome but also in discovering a spasmodic incident caused by a sudden attack during sleep. As mentioned above, 
a typical respiration detecting method during sleep is a method of detecting respiration based on time base measured 
values of a load sensor or a pressure-sensitive sensor. Additionally, a method of using a vibration sensor, a radio-wave 

35 sensor, or air-pressure detection is publicly known (e.g., JP-A-H7-327939, JP-A-H1 1-281 95, and JP-A-2000-83927). 
In these methods, since a measured signal is weak, a high-performance signal amplifier or some kind of signal process- 
ing is required to acquire and detect a stable signal, and, as a system, it becomes expensive and large in scale. 
[0005] On the other hand, some proposals have been also made of a method of acquiring an image of a sleeping 
person by use of an image pickup device and detecting respiration based on the acquired image. With recent devel- 

40 opments in electronic equipment, a high-performance image pickup device has appeared on the market at an extremely 
low price, and, since the device has noncontact properties, the method of detecting respiration based on an image has 
been brought to public attention as a technique having high practical usefulness. 

[0006] For example, in "Image-processing device and patient-observing device" of JP-A-H1 1-86002 and "Region- 
of-interest setting device of respiration monitoring and respiration-monitoring system" of J P-A-H1 1-225997, the basic 
45 features of those inventions is to monitor respiration by examining a difference between images acquired in different 
time by the image pickup device. 

[0007] The "image-processing device and patient-observing device" of JP-A-H1 1 -86002 is composed of a TV cam- 
era, a respiration monitoring device, and a local-region automatic setting processing device. The local-region automatic 
setting processing device is composed of an edge detecting section that detects an edge included in a local region 

so imageset on an image forwhich a patient who requires care is photographed by theTV camera, a brightness distribution 
measuring section that measures the brightness distribution of each local partial region image divided by the edge in 
the local region image, and a determining section that sets a local region image to extract movement information by 
analyzing information about a detected edge and information about a measured brightness distribution. The local region 
image is divided into a plurality of local partial regions that are identical in brightness, and a time differentiation process 

55 is applied to each pixel included in each local partial region, and the total amount of the time differentiation is calculated. 
A time base change of this amount is analyzed, and a periodic appearance is detected as respiration, whereas irreg- 
ularities in the period and amplitude are detected as great body movements, such as a body twist. 
[0008] In the "Region-of-interest setting device of respiration monitoring and respiration-monitoring system" of 
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JP-A-H1 1 -225997, a calculation is first performed of the absolute value of a difference between every one frame of a 
plurality of frame images picked up by a CCD camera over 1/2 periods of respiration. Thereafter, the difference images 
are integrated and stored, the positions and sizes of variation regions are then calculated from variation information 
that has been integrated and stored, and they are set as temporary regions in order from the largest to the smallest 

s region in the variation ones. Thereafter, a judgment is made of whether a concentration-value histogram, that shows 
the distribution of the number of pixels of each concentration value , exhibits a two-peak characteristic having a height 
greater than a predetermined value in the temporary regions and whether the area value of the variation regions is 
greaterthan a given value. If this condition is satisfied, the temporary region is set as a region of interest (abbreviated 
as ROI). Further, a time differentiation process is performed in the set ROI, the absolute value of the difference between 

10 each pixel is then obtained, and a surface integral is applied. The surface integral is performed in a time series manner, 
and, like the invention of JP-A-H1 1-86002 a time base change in this surface integral is analyzed, and a periodic 
appearance is detected as respiration, whereas irregularities in the period and amplitude are detected as great body 
movements, such as a body twist. 

[0009] Further, a method of detecting respiration by an optical flow of movement of a sleeping person is known as 
is an image-using technique other than the aforementioned methods. The optical flow is characterized by detecting the 
movement of a sleeping person as a velocity vector, and a respiration waveform having a periodic rhythm and a body- 
movement waveform having a high peak can be obtained from a vector field by employing the fact that most upward 
vectors are detected in inspiration whereas most downward vectors are detected in expiration. 
[0010] The respective methods described above are to observe the movement of shadows on a quilt by use of 
20 illumination light, and there remains the fundamental problem of being sometimes incapable of detecting the movement 
of shadows depending on lighting conditions, the posture of a sleeping person, or the design of the quilt. Additionally, 
since an image pickup device must be set close to the sleeping person in order to photograph the shadow on the quilt, 
the respiration monitoring according to the aforementioned methods is considered to entail a psychologically overpow- 
ered feeling when the person to be monitored goes to bed. 
25 [001 1] Additionally, it is said that the method based on a time differentiation can evaluate a frequency of the movement 
of a targeted person, but cannot make a quantitative evaluation of the movement thereof. In contrast, in the optical- 
flow method, the optical flow enables a quantitative evaluation of the movement of a sleeping person, but, in practice, 
much computation time is needed to calculate the optical flow, and there remains the problem of requiring expensive 
processing equipment. 

Disclosure of Invention 

[0012] Therefore, the invention of this application has been made in consideration of the foregoing circumstances, 
and it is an object of the invention to provide a monitor capable of detecting respiration of a sleeping person without 
35 being affected by the posture of the sleeping person or the indoor illumination light and capable of easily evaluating 
detected respiration quantitatively through image measurement. 

[0013] The monitor of the present invention comprises lighting pattern projecting means for projecting a specified 
lighting pattern, image pickup means for picking up light of projected wavelength continuously, moving amount calcu- 
lating means for calculating an inter-frame moving amount of the lighting pattern from two frames of images acquired 

40 by the image pickup means at different times, moving amount waveform generating means for generating moving 
amount waveform data comprising inter-frame moving amounts arranged in time series, and waveform detecting means 
for detecting a movement of an object to be picked up from the moving amount waveform data. 
[0014] The waveform detecting means is to detect the body movement and respiration of a sleeping person, and the 
monitor further comprises safety deciding means for deciding the safety of the sleeping person from the body movement 

45 and respiration of the sleeping person and signal outputting means for, when the safety deciding means decides that 
the sleeping person is in a dangerous state, outputting a signal indicating it. 

[0015] The moving amount calculating means can easily calculate the moving amount of the lighting pattern by 
calculating the inter-frame moving amount of the lighting pattern in an axial direction connecting the lighting pattern 
projecting means and the image pickup means. 
so [0016] The waveform detecting means can detect a respiratory pattern and a body movement pattern by detecting 
a periodic pattern as the respiratory pattern and by detecting a variation having a high peak as the body movement 
pattern from moving-amount waveform data. 

[0017] The safety deciding means can decide a dangerous state of a sleeping person by deciding that the sleeping 
person is in a dangerous state when a period of the respiratory pattern falls into disarray in a short time, when the 
55 period of the respiratory pattern suddenly changes, when the respiratory pattern continuously disappears, or when the 
body movement pattern frequently appears in a short time. 

[0018] The moving amount calculating means can easily and reliably detect the movement of an object to be pho- 
tographed by calculating an inter-frame moving amount of the lighting pattern in an axial direction connecting the 
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lighting pattern projecting means and the image pickup means while providing a positive or negative sign in accordance 
with a moving direction. 

[0019] The waveform detecting means can reliably detect a respiratory pattern by, from among respiratory patterns, 
detecting a value having a positive or negative sign corresponding to a movement in a direction from the lighting pattern 
s projecting means to the image pickup means on an image and a value having a positive or negative sign corresponding 
to a movement in a direction from the image pickup means to the lighting pattern projecting means on the image as 
an expiratory pattern and as an inspiratory pattern, respectively, or vice versa. 

[0020] The waveform detecting means can well adapt to processing in a computer by counting zero crosses where 
signs are reversed between an expiratory pattern and an inspiratory pattern to count respirations. 

10 [0021] The waveform detecting means can further exactly calculate a respiration number per unit time or a respiration 
period by calculating the respiration number per unit time or the respiration period from the counted respirations. 
[0022] The waveform detecting means can further easily and exactly calculate a respiration number per unit time or 
a respiration period by conducting a frequency analysis of moving-amount waveform data or sampling data obtained 
by sampling respiratory patterns, by calculating the most prominent frequency from a resulting frequency spectrum, 

is and by calculating the respiration number per unit time or the respiration period from this frequency. 

[0023] The waveform detecting means can further calculate the respiration number per unit time or the respiration 
period through digital signal processing by conducting a frequency analysis with discrete Fourier transform or discrete 
wavelet transform. 

[0024] The safety deciding means can reliably decide that a sleeping person is in a dangerous state by deciding 
20 whether respiration belongs to normal respiration, Cheyne-Stokes respiration, central hyperventilation, ataxic respira- 
tion, or Kussmaul respiration from expiratory patterns and inspiratory patterns and by deciding that the sleeping person 
is in a dangerous state when the respiration belongs to any one of Cheyne-Stokes respiration, central hyperventilation, 
ataxic respiration, and Kussmaul respiration. 

[0025] The moving amount calculating means calculates an inter-frame moving amount in a partial region of the 
25 lighting pattern in an axial direction connecting the lighting pattern projecting means and the image pickup means, and 
determines a region in an image where an inter-frame moving amount in the partial region appears much greater than 
a predetermined amount or where an inter-frame movement in the partial region appears frequently beyond a prede- 
termined amount as a region where the sleeping person exists, and the safety deciding means can safely and reliably 
decide that the sleeping person is in a dangerous state by deciding that the sleeping person is in a dangerous state 
30 when the region where the sleeping person exists moves in a shorter time than a predetermined time and moves 
frequently beyond a predetermined amount. 

[0026] The moving amount calculating means calculates an inter-frame moving amount in a partial region of the 
lighting pattern in an axial direction connecting the lighting pattern projecting means and the image pickup means, and 
determines a region in an image where an inter-frame moving amount in the partial region appears much greater than 
35 a predetermined amount or where an inter-frame movement in the partial region appears more frequently than a pre- 
determined frequency as a region where the sleeping person exists, and the safety deciding means can reliably decide 
that the sleeping person is in a state of imminently falling from a bed by deciding that the sleeping person is in a 
dangerous state when the region where the sleeping person exists is brought closer to one end of the bed than a 
predetermined distance. 

40 [0027] Additionally, it is possible to know that the sleeping person is in a dangerous state at a location remote there- 
from by providing informing means for, based on a signal output from signal outputting means, informing a third party 
that the sleeping person is in a dangerous state. 

[0028] The informing means can urge a third party to swiftly deal with the situation by informing the third party that 
the sleeping person is in a dangerous state by voice, characters, symbols, intensity of light including interior illumination 

45 light, or vibrations through a telecommunication line. 

[0029] The lighting pattern projecting means projects a lighting pattern that has a plurality of lighting spots, and the 
moving amount calculating means calculates an inter-frame moving amount of each lighting spot in an axial direction 
connecting the lighting pattern projecting means and the image pickup means while providing a positive or negative 
sign in accordance with amoving direction, and calculates an inter-frame moving amount of the lighting pattern by use 

so of a moving amount of one or more of the lighting spots, whereby the moving amount of the lighting pattern can be 
easily calculated. 

[0030] The lighting pattern projecting means projects a lighting pattern that has a plurality of lighting spots, and the 
moving amount calculating means calculates an inter-frame moving amount of each lighting spot in an axial direction 
connecting the lighting pattern projecting means and the image pickup means while providing a positive or negative 
55 sign in accordance with a moving direction, and calculates the total moving amount of each lighting spot as an inter- 
frame moving amount of the lighting pattern, whereby a statistical noise influence can be reduced. 
[0031] The lighting pattern projecting means projects a lighting pattern that has a single slit ray or a plurality of slit 
rays, and the moving amount calculating means calculates an inter-frame moving amount of each lighting spot in an 
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axial direction connecting the lighting pattern projecting means and the image pickup means while providing a positive 
or negative sign in accordance with a moving direction, and calculates an inter-frame moving amount of the lighting 
pattern by use of a moving amount of a pixel corresponding to the single or plural slit rays, whereby the moving amount 
of the lighting pattern can be easily calculated. 

s [0032] The lighting pattern projecting means projects a lighting pattern that has a single slit ray or a plurality of slit 
rays, and the moving amount calculating means calculates an inter-frame moving amount of each pixel corresponding 
to a slit ray in an axial direction connecting the lighting pattern projecting means and the image pickup means while 
providing a positive or negative sign in accordance with a moving direction, and calculates the total moving amount of 
each pixel corresponding to the slit ray as an inter-frame moving amount of the lightingpattern, whereby the number 

10 of measurement points can be increased, and a statistical noise influence can be reduced. 

[0033] The lighting pattern projecting means is disposed directly above a part close to one end of an edge of bedding 
facing a head of or a foot of a person sleeping on the bedding, and the image pickup means is disposed directly above 
a part close to an opposite end thereof, whereby monitoring can be performed without allowing the sleeping person to 
have an intense consciousness of being monitored. 

is [0034] The provision of presence-in-bed detecting means for detecting the presence/absence of a sleeping person 
makes it possible to swiftly and accurately judge whether the sleeping person is absent or the sleeping person has 
stopped his/her breathing. 

[0035] The provision of a pressure sensitive switch that can be placed under the sleeping person and can be used 
to detect the presence/absence of the sleeping person makes it possible to swiftly and accurately judge whether the 
20 sleeping person is absent or the sleeping person has stopped his/her breathing. 

[0036] This specification includes the contents of the description and/or the drawings of Japanese Patent Application 
No. 2000-372820 on which the priority of this application is based. 

Brief Description of the Drawings 

25 

[0037] 

Fig. 1 Is a schematic diagram showing the structure of a monitor according to the invention of this application; 
Fig. 2 is a schematic diagram explaining the calculation of an inter-frame moving amount of a lighting pattern on 
30 an image in a moving amount calculating means; 

Fig. 3 is a schematic diagram showing the principle of the monitor according to the invention of this application; 
Fig. 4 shows an example of a waveform pattern of moving-amount waveform data; 

Fig. 5 is a schematic diagram showing the principle of the respiration detection of the monitor according to the 
invention of this application; 

35 Fig. 6(a) , Fig. 6(b), Fig. 6(c), and Fig. 6(d) show examples of normal and abnormal respiratory patterns; 

Fig. 7 is a schematic diagram showing a lighting pattern projected by a lighting pattern projecting means; 
Fig. 8 is a schematic diagram showing a lighting pattern projected by the lighting pattern projecting means; 
Fig. 9 is a schematic diagram showing an arrangement of the lighting pattern projecting means and an image 
pickup means; 

40 Fig. 10 is a schematic diagram showing an arrangement of the lighting pattern projecting means and the image 

pickup means; 

Fig. 11 is a schematic diagram showing an arrangement of the lighting pattern projecting means and the image 
pickup means; 

Fig. 12 is a schematic diagram showing the structure of an indoor safety monitor that is the invention of this ap- 
45 plication; and 

Fig. 13 is a schematic diagram showing the structure of an indoor safety monitor that is the invention of this ap- 
plication. 

Best Mode for Carrying Out the Invention 

50 

[0038] The invention of this application has the aforementioned features, and embodiments thereof will hereinafter 
be described. 

[0039] Fig. 1 is a schematic diagram showing the structure of a monitor according to the invention of this application. 
The monitor 6 is composed of a lighting pattern projecting means 1, an image pickup means 5, a moving amount 
55 calculating means 7, a moving amount waveform generating means 8, a waveform detecting means 9, a safety deciding 
means 10, a signal outputting means 11, and an informing means 12. A lighting pattern 4 is first projected from the 
lighting pattern projecting means 1 onto a body 2 of a sleeping person or onto bedding 3. It is preferable to set the 
wavelength of projected light to be infrared rays, so as to project without allowing the sleeping person to have con- 
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sciousness of being monitored. The lighting pattern 4 projected to the body 2 or the bedding 3 is continuously picked 
up as an image by the image pickup means 5. The image pickup means 5 can pick up, for example, an infrared ray 
that is a wavelength of the projected light. In response to a movement in an optical axial direction of the image pickup 
means resulting from a movement of the body 2 or resulting from a movement of the bedding 3 responding to the 
5 movement of the body 2, a movement of a lighting pattern, whose optical axis is different from that of the image pickup 
means, in an pickup image plane occurs, and an inter-frame moving amount of this lighting pattern is calculated by 
the moving amount calculating means 7 from two frames of images different in time that have been acquired by the 
image pickup means 5. 

[0040] Thereafter, in the moving amount waveform generating means 8, moving amount waveform data is generated 
10 by arranging the inter-frame moving amounts calculated by the moving amount calculating means 7 in time series. 
Further, in the waveform detecting means 9, the body movement and respiration of the sleeping person are detected 
from the moving amount waveform data. Even 2 or 3 frames/second of images to be acquired are enough to generate 
the waveform of respiration. Preferably, the images are acquired at regular intervals in order to generate the moving 
amount waveform data. 

15 [0041 ] The monitor 6 of the invention of this application may be provided with the safety deciding means 1 0 by which 
the safety of a sleeping person is decided from the body movement and respiration of the sleeping person detected 
by the waveform detecting means 9. When it is decided that the sleeping person is in a dangerous state in the safety 
deciding means 10, the signal outputting means 11 outputs a signal indicating it. As a result, the dangerous state can 
be recorded, for example, together with its time. Further, based on the signal output by the signal outputting means 

20 11, the informing means 12 automatically informs a third party the sleeping person is in a dangerous state. 

[0042] Fig. 2 is a schematic diagram explaining the calculation of an inter-frame moving amount of a lighting pattern 
on an image in the moving amount calculating means. In the moving amount calculating means 7, an inter-frame 
moving amount of the lighting pattern in an axial direction connecting the lighting pattern projecting means 1 and the 
image pickup means 5 is calculated from two frames different in time among images acquired by the image pickup 

25 means 5. 

[0043] Fig. 3 is a schematic diagram showing the principle of the monitor according to the invention of this application. 
In an optical arrangement shown in Fig. 3, the following equation is established. 

30 6=dlZ/{h(h-Z) } 

where 5: inter-frame moving amount of a lighting pattern, 

h: installation heights of the lighting pattern projecting means and the image pickup means, 
d: distance between the lighting pattern projecting means and the image pickup means, 

35 I: focal length of the image pickup means, and 

Z: displacement amount in the height direction. 
[0044] That is, the inter-frame moving amount 6 of the lighting pattern corresponds to a displacement amount Z in 
the height direction resulting from a movement of the body 2 of a sleeping person or resulting from a movement of 
bedding 3 in response to the movement of the body 2. Therefore, in the waveform detecting means 9, a waveform 

40 showing a periodic pattern can be detected as a respiratory pattern, and a waveform showing a variation having a high 
peak can be detected as a body movement pattern (i.e., pattern in which the waveform is varied by a body movement, 
such as turning over in bed) separately from each other in the moving amount waveform data generated by the moving 
amount waveform generating means 8. In the respiratory pattern, a respiration number per unit time can be known 
from its periodicity. Further, the time transition of the periodicity shows the stability of the respiration of the sleeping 

45 person. Further, since the inter-frame moving amount of the lighting pattern corresponds to a displacement amount in 
the height direction as mentioned above, it serves also as a means for knowing the depth of the respiration. 
[0045] A predominant frequency component can be calculated by conducting a frequency analysis of moving amount 
waveform data or respiratory pattern with discrete Fourier transform or discrete wavelet transform, and a respiration 
number per unit time can also be calculated from the value of this frequency. Fast Fourier transform can be mentioned 

so as a typical example of discrete Fourier transform. For example, on the assumption that the most prominent sample 
in a result obtained by applying fast Fourier transform to the respiratory pattern is the M-th sample in sampling data in 
which N samples of respiratory patterns have been sampled at sampling intervals of Q second, the most prominent 
frequency f can be obtained by M/(QxN), and a respiration number per minute can be obtained by 60 xf. A respiration 
period can be obtained by 1+f. As a matter of course, various frequency analysis techniques other than fast Fourier 

55 transform can be applied in the invention of this application. 
[0046] In the safety deciding means 1 0, for example, 



(i) when the period of a respiratory pattern falls into disarray in a short time, 
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(ii) when the period of a respiratory pattern suddenly changes, 

(iii) when a respiratory pattern continuously disappears, and 

(iv) when a body movement pattern frequently appears in a short time, 

s it is decided that a sleeping person is in a dangerous state. The condition of (i) or (ii) is considered to be caused by an 
infirmity of the lungs, such as spontaneous pneumothorax or bronchial asthma, a cardiopathy, such as congestive 
heart failure, or a cerebrovascular disease, such as cerebral hemorrhage. The condition of (iii) appears when respiration 
stops. Apossible condition of (iv) is the fact that the sleeping person suffers from some reason and is in distress. 
[0047] In the moving amount calculating means 7, preferably, an inter-frame moving amount of a lighting pattern in 

10 an axial direction connecting the lighting pattern projecting means 1 and the image pickup means 5 is calculated from 
two frames of images different in time while applying a positive or negative sign in accordance with a moving direction. 
Hereby, concerning the respiration, an expiratory pattern and an inspiratory pattern shown in Fig. 4 are always obtained, 
and the respiration can be counted by measuring the number of "zero crosses" (intersections where the sign is reversed) 
appearing from the expiratory pattern to the inspiratory pattern or from the inspiratory pattern to the expiratory pattern. 

is This is advantageous when processing is performed by a computer. 

[0048] Fig. 4 shows an example of a waveform pattern of moving-amount waveform data. The moving amount wave- 
form data generated by the moving amount waveform generating means 8 shows a waveform pattern, for example, 
shown in Fig. 4. 

[0049] Fig. 5 is a schematic diagram showing the principle of the respiration detection of the monitor according to 
20 the invention of this application. In an optical arrangement shown in Fig. 5, when the height changes upward "A, " a 
lighting pattern on an image moves in a direction "A"' from the image pickup means to the lighting pattern projecting 
means. In contrast, when the height changes downward "B," the lighting pattern on the image moves in a direction "B 1 " 
from the lighting pattern projecting means to the image pickup means on the image. 

[0050] Therefore, concerning the inter-frame moving amount of the lighting pattern, if moving amount wave form 
25 data is generated from the inter-frame moving amount calculated while having a positive or negative sign in accordance 
with the moving direction, and if a waveform showing a periodic pattern among the moving amount waveform data is 
detected as a respiratory pattern, a part 41 having a sign corresponding to a movement in the direction "A"' can be 
detected as an expiratory pattern, whereas a part 42 having a sign corresponding to a movement in the direction "B' " 
can be detected as an inspiratory pattern among the detected respiratory patterns. 
30 [0051] Since the moving direction of a lighting pattern follows a horizontal or vertical direction in the picked-up image 
by setting the horizontal or vertical direction of a picked-up image to coincide with the axial direction connecting the 
lighting pattern projecting means and the image pickup means, the moving amount of the lighting pattern can be easily 
calculated. 

[0052] Further, a respiration number per unit time (or respiration period) can be known by a time point where the 
35 part 41 having a sign corresponding to a movement in the direction "A"' and the part 42 having a sign corresponding 
to a movement in the direction "B"' each appear once as a single breath. 

[0053] In the safety deciding means, a decision may be made as to whether the respiration of a sleeping person is 
a normal respiration by pre-storing each respiratory pattern concerning normal and abnormal respirations and com- 
paring these with a respiratory pattern of the sleeping person. 

40 [0054] Fig. 6 (a) , Fig. 6 (b) , Fig. 6 (c) , and Fig. 6 (d) show examples of normal and abnormal respiratory patterns. 
A normal respiratory pattern to be registered is a periodic pattern shown in Fig. 6(a). On the other hand, a respiratory 
pattern, such as Cheyne-Stokes respiration, central hyperventilation, ataxic respiration, or Kussmaul respiration, that 
is considered to occur when a physiologically disorder arises in the body, is registered as an abnormal respiratory 
pattern. As an example, a respiratory pattern of Cheyne-Stokes respiration is shown in Fig. 6(b), a respiratory pattern 

45 of central hyperventilation is shown in Fig. 6(c), and a respiratory pattern of ataxic respiration is shown in Fig. 6(d). 
These respiratory patterns are obviously different in the waveform from the normal respiratory pattern, and a decision 
is made as to a respiratory pattern to which that of a sleeping person belongs on the basis of the fact that they are 
different in the frequency of respiration, the occurrence count thereof, and the depth thereof. 
[0055] When it is decided that the respiration of the sleeping person belongs to a respiratory pattern that is considered 

so to occur when physiologically disorders arise in the body, the safety deciding means decides that the sleeping person 
is carrying out abnormal respiration and is in a dangerous state. When the sleeping person displays abnormal respi- 
ration, the informing means may inform a third party about the name of the respiratory pattern of the sleeping person, 
the name of a disease considered to cause the respiration, the disease organ, the disease part, etc. Table 1 shows 
the disease name or disease part when the abnormal respiratory pattern occurs. 

55 

Table 1 

I Cheyne-Stokes respiration I Disorder under both-sides cerebral cortex and of diencephalon I 
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Table 1 (continued) 



Central hyperventilation 



Disorder from lower midbrain to upper pons 



Ataxic respiration 



Disorder from lower pons to upper medulla oblongata 



5 



Kussmaul respiration 



Diabetic coma or uremia 



[0056] Fig. 7 and Fig. 8 are each a schematic diagram showing a lighting pattern projected by the lighting pattern 
projecting means. Preferably, a lighting pattern projected by the lighting pattern projecting means has a spatially discrete 
10 distribution. For example, a plurality of lighting spots 71 shown in Fig. 7, a single slit ray 81 shown in Fig. 8, or a plurality 
of slit rays 82 are selected and used. 

[0057] In a lighting pattern having a plurality of lighting spots, an inter-frame moving amount of each lighting spot is 
calculated while adding a positive or negative sign in accordance with amoving direction, and the total moving amount 
of each lighting spot is calculated as an inter-frame moving amount of the lighting pattern. Likewise, in a lighting pattern 
15 having a single or a plurality of slit rays, an inter-frame moving amount of each pixel corresponding to the slit ray is 
calculated while adding a positive or negative sign in accordance with a moving direction, and the total moving amount 
of each pixel corresponding to the slit ray is calculated as an inter-frame moving amount of the lighting pattern. A 
statistical noise influence can be reduced by calculating the total inter-frame moving amount when spatially discrete 
lighting patterns are used. 

20 [0058] In the moving amount calculating means, a region in an image where the inter-frame moving amount of a 
lighting pattern appears greatly or where an inter-frame movement of the lighting pattern appears frequently is deter- 
mined as a region where the sleeping person exists. When this region frequently moves in a short time, the safety 
deciding means may decide that the sleeping person is in a dangerous state. This can beregarded as a condition where 
the sleeping person suffers from some reason and is in distress. 

25 [0059] The safety deciding means may decide that the sleeping person is in a dangerous state when the region 
where the sleeping person exists is brought extremely close to one of the sides of a bed and when the sleeping person 
on the bed is being monitored. This can be regarded as a situation where the sleeping person is in a dangerous position 
as if to fall from the bed. 

[0060] In the monitor of the invention of this application, preferably, a range where the lighting pattern is projected 

30 is set within a range covering positions that can be occupied by the belly, chest, back, and shoulders of the sleeping 
person. Likewise, preferably, a range of a region photographed by an image pickup device is set within a rangecovering 
positions that can be occupied by the belly, chest, back, and shoulders of the sleeping person. 
[0061] Fig. 9 and Fig. 1 0 are each a schematic diagram showing an arrangement of the lighting pattern projecting 
means and the image pickup means. For example, as shown in Fig. 9, the lighting pattern projecting means 91 is 

35 disposed directly above a part close to the head 92 or the foot 93 of the sleeping person, and the image pickup means 
94 is disposed directly above a part around the belly 95 of the sleeping person. At an edge 104 of bedding 101 placed 
under the sleeping person facing the head 92 or the foot 93 of the sleeping person as shown in Fig. 10, the lighting 
pattern projecting means 106 may be disposed directly above a part close to one end 105 of the edge 104, and the 
image pickup means 1 08 may be disposed directly above a part close to an opposite end 1 07 of the edge 1 04. The 

40 arrangement shown in Fig. 9 can capture the respiration of the sleeping person extremely sensitively, and has the 
advantage of being able to detect a detailed respiratory pattern. However, since the image pickup device comes into 
view of the sleeping person, there can be a case in which the arrangement is accompanied by a psychologically op- 
pressive feeling. In contrast, an arrangement shown in Fig. 10 cannot specifically catch the respiration unlike the ar- 
rangement shown in Fig. 9, but this arrangement is considered to have a weaker psychologically oppressive feeling. 

45 Of course, the arrangement shown in Fig. 9 can be formed to sacrifice slightly the sensitivity of respirations as in Fig. 
11 , and an oppressive feeling may be reduced by disposing the lighting pattern projecting means 111 and the image 
pickup means 1 1 2 at one side end 1 1 3 of the bedding. These arrangements are to be appropriately selected depending 
on the situation. 

[0062] Any type of lighting pattern projecting means can be used if the lighting pattern projecting means can project 
so light spatially discretely. Use can be made of, for example, a fiber grating, a diffraction grating, a lens array, a formed 
image of a light source array or an aperture array, a device that collimates an outgoing beam of the light source array 
or the aperture array, etc. 

[0063] In the monitor of the invention of this application, as an auxiliary means for accurately detecting whether a 
sleeping person is present or absent, a pressure sensitive switch may be disposed in the bedding placed under the 
55 sleeping person. The presence/absence of the sleeping person is determined by ON/OFF of this switch. 

[0064] Further, in the monitor of the invention of this application, the informing means has a voice outputting function 
and informs a third party by voice that the sleeping person is in a dangerous state. It may inform the third party by 
characters, symbols, intensity of light including interior illumination light, or vibrations. Further, the informing means 
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may have a function to connect to a telecommunications line, such as a general telephone line, ISDN line, PHS line, 
or cellulartelephone line, and may inform the third party by voices, characters, or symbols that the sleeping person is 
in a dangerous state. 

[0065] Fig. 12 is a schematic diagram showing an arrangement of the lighting pattern projecting means and the 
s image pickup means. The principle of the invention of this application is not applied limitedly to a monitor for a sleeping 
person, of course. For example, as a matter of course, the principle can be applied also to an indoor safety monitor 
that monitors the safety of a person in a room. The indoor safety monitor has the same structure as the monitor of the 
invention of this application, and is shown, for example, in Fig. 12. 

[0066] First, a lighting pattern 123 is projected from a lighting pattern projecting means 121 to a body 122 of a person 
10 to be monitored. The lighting pattern 1 23 projected to the body 1 22 is continuously picked up as an image by an image 
pickup means 124, and, from two frames of images different in time acquired by the image pickup means 124, an inter- 
frame moving amount of the lighting pattern 123 resulting from a movement of the body 122 is calculated by the moving 
amount calculating means. 

[0067] Thereafter, in the moving amount waveform generating means, moving amount waveform data is generated 
15 by arranging the inter-frame moving amounts calculated by the moving amount calculating means in time series. Fur- 
ther, in the waveform detecting means, a body movement and respiration of the person to be monitored are detected 
from the moving amount waveform data. 

[0068] The indoor safety monitor is provided with a safety deciding means. From the body movement and respiration 
of the person detected by the waveform detecting means, the safety deciding means decides whether the person to 

20 be monitored is safe or not. When the safety deciding means decides that the person is in a dangerous state, the 
informing means automatically informs a third party the person to be monitored is in a dangerous state. 
[0069] In the moving amount calculating means, an inter-frame moving amount of a lighting pattern on an image in 
an axial direction connecting the lighting pattern projecting means 121 and the image pickup means 124 is calculated 
from two frames different in time among images acquired by the image pickup means 124. 

25 [0070] In the waveform detecting means, a waveform indicating a periodic pattern is detected as a respiratory pattern, 
whereas a waveform indicating a variation having a high peak is detected as a body movement pattern among the 
moving amount waveform data generated by the moving amount waveform generating means. 
[0071] The safety deciding means decides that a time zone where a body-movement pattern of the person to be 
monitored disappears indicates a state where the person stops moving, and, in this time zone, the safety deciding 

so means decides that the person is in a dangerous state when the period of the respiratory pattern falls into disarray in 
a short time, when the period of the respiratory pattern suddenly changes, or when the respiratory pattern continuously 
disappears. 

[0072] The indoor safety monitor mentioned above works effectively especially when a monitoring region is limited. 
For example, this is extremely adaptable for monitoring a person having a bath as shown in Fig. 13. Likewise, this 
35 works effectively to monitor the safety of a person in a bathroom. 

[0073] The present invention is not limited to the aforementioned embodiments. 

[0074] It is permissible to project light that has been subjected to amplitude modulation and extract a modulating 
signal from a picked-up signal. Thereby, it is possible to reduce an influence by which general light, such as indoor 
illumination light, is changed into a noise. 
40 [0075] If precision is sacrificed to some degree, the frame may be a field. Therefore, the frame set forth in the ap- 
pended claims has a broad concept including a field. 

Industrial Applicability 

45 [0076] As described in detail in the foregoing description, according to the invention of this application, it is possible 
to provide a monitor by image measurement capable of reliably detecting the respiration of a sleeping person and 
easily making a quantitative evaluation of the detected respiration without being influenced by the posture of the sleep- 
ing person, illumination light, and design of a quilt. 

[0077] The invention of this application is strongly expected to be put into practical use as a technique that can 
so support the achievement of swift emergency treatment when a senior citizen or sick person falls into a crisis condition. 
[0078] All publications, patents, and patent applications cited in the present specification are to be employed in the 
present specification as references without any changes. 



55 Claims 



1. A monitor comprising: 
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lighting pattern projecting means for projecting a specified lighting pattern; 
image pickup means for picking up light of a projected wavelength continuously; 

moving amount calculating means for calculating an inter-frame moving amount of the lighting pattern from 
two frames of images acquired by the image pickup means at different times; 
s moving amount waveform generating means for generating moving amount waveform data comprising inter- 

frame moving amounts arranged in time series, and 

waveform detecting means for detecting a movement of an object to be picked up from the moving amount 
waveform data. 

10 2. The monitor as set forth in Claim 1 , wherein the waveform detecting means detects a body movement and respi- 
ration of a sleeping person, the monitor further comprising: 

safety deciding means for deciding the safety of the sleeping person from the body movement and respiration 
of the sleeping person; and 

15 signal outputting means for, when the safety deciding means decides that the sleeping person is in a dangerous 

state, outputting a signal indicating it. 

3. The monitor as set forth in Claim 2, wherein the moving amount calculating means calculates an inter-frame moving 
amount of a lighting pattern in an axial direction connecting the lighting pattern projecting means and the image 

20 pickup means. 

4. The monitor as set forth in Claim 2 or 3, wherein the waveform detecting means detects a periodic pattern as a 
respiratory pattern, and detects a variation having a high peak as a body movement pattern from the moving 
amount waveform data. 

25 

5. The monitor as set forth in Claim 4, wherein the safety deciding means decides that the sleeping person is in a 
dangerous state when a period of the respiratory pattern falls into disarray in a short time, when the period of the 
respiratory pattern suddenly changes, when the respiratory pattern continuously disappears, or when the body 
movement pattern frequently appears in a short time. 

30 

6. The monitor as set forth in Claim 4 or 5, wherein the moving amount calculating means calculates an inter-frame 
moving amount of the lighting pattern in an axial direction connecting the lighting pattern projecting means and 
the image pickup means while providing a positive or negative sign in accordance with a moving direction. 

35 7. The monitor as set forth in Claim 6, wherein from among respiratory patterns, the waveform detecting means 
detects a value having a positive or negative sign corresponding to a movement in a direction from the lighting 
pattern projecting means to the image pickup means on an image and a value having a positive or negative sign 
corresponding to a movement in a direction from the image pickup means to the lighting pattern projecting means 
on the image as an expiratory pattern and as an inspiratory pattern, respectively, or vice versa. 

40 

8. The monitor as set forth in Claim 7, wherein the waveform detecting means counts zero crosses where signs are 
reversed between an expiratory pattern and an inspiratory pattern and counts respirations. 

9. The monitor as set forth in Claim 8, wherein the waveform detecting means calculates a respiration number per 
45 unit time or a respiration period from the counted respirations. 

10. The monitor as set forth in Claim 1 or 7, wherein the wave form detecting means calculates a respiration number 
per unit time or a respiration period by conducting a frequency analysis of the moving-amount waveform data set 
forth in Claim 1 or the sampling data obtained by sampling respiratory patterns set forth in Claim 7, by calculating 

so the most prominent frequency from a resulting frequency spectrum, and by calculating the respiration number per 

unit time or the respiration period from this frequency. 

11. The monitor as set forth in Claim 10, wherein the waveform detecting means conducts a frequency analysis with 
discrete Fourier transform or discrete wavelet transform. 

55 

12. The monitor as set forth in any one of Claims 7 through 9, wherein the safety deciding means decides whether 
respiration belongs to normal respiration, Cheyne-Stokes respiration, central hyperventilation, ataxic respiration, 
or Kussmaul respiration from expiratory patterns and inspiratory patterns and decides that the sleeping person is 
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in a dangerous state when the respiration belongs to any one of Cheyne-Stokes respiration, central hyperventila- 
tion, ataxic respiration, and Kussmaul respiration. 

13. The monitor as set forth in Claim 2, wherein the moving amount calculating means calculates an inter-frame moving 
s amount in a partial region of the lighting partem in an axial direction connecting the lighting pattern projecting 

means and the image pickup means, and determines a region in an image where an inter-frame moving amount 
in the partial region appears much greater than a predetermined amount or where an inter-frame movement in the 
partial region appears frequently beyond a predetermined amount as a region where the sleeping person exists, 
and the safety deciding means decides that the sleeping person is in a dangerous state when the region 
10 where the sleeping person exists moves in a shorter time than a predetermined time and moves frequently beyond 

a predetermined amount. 

14. The monitor as set forth in Claim 2, wherein the moving amount calculating means calculates an inter-frame moving 
amount in a partial region of the lighting pattern in an axial direction connecting the lighting pattern projecting 

is means and the image pickup means, and determines a region in an image where an inter-frame moving amount 

in the partial region appears much greater than a predetermined amount or where an inter-frame movement in the 
partial region appears more frequently than a predetermined frequency as a region where the sleeping person 
exists, 

and the safety deciding means decides that the sleeping person is in a dangerous state when the region 
20 where the sleeping person exists is brought closer to one end of the bed than a predetermined distance. 

15. The monitor as set forth in any one of Claims 2 through 14, comprising informing means for, based on a signal 
output from signal outputting means, informing a third party that the sleeping person is in a dangerous state. 

25 16. The monitor as set forth in Claim 15, wherein the informing means informs the third party that the sleeping person 
is in a dangerous state by voices, characters, symbols, intensity of light including interior illumination light, or 
vibrations through a telecommunications line. 

17. The monitor as set forth in any one of Claims 1 through 16, wherein the lighting pattern projecting means projects 
30 a lighting pattern that has a plurality of lighting spots, 

and the moving amount calculating means calculates an inter-frame moving amount of each lighting spot in 
an axial direction connecting the lighting pattern projecting means and the image pickup means while providing a 
positive or negative sign in accordance with a moving direction, and calculates an inter-frame moving amount of 
the lighting pattern by use of a moving amount of one or more of the lighting spots. 

35 

18. The monitor as set forth in any one of Claims 1 through 16, wherein the lighting pattern projecting means projects 
a lighting pattern that has a plurality of lighting spots, 

and the moving amount calculating means calculates an inter-frame moving amount of each lighting spot in 
an axial direction connecting the lighting pattern projecting means and the image pickup means while providing a 
40 positive or negative sign in accordance with a moving direction, and calculates the total moving amount of each 

lighting spot as an inter-frame moving amount of the lighting pattern. 

19. The monitor as set forth in any one of Claims 1 through 16, wherein the lighting pattern projecting means projects 
a lighting pattern that has a single slit ray or a plurality of slit rays, and the moving amount calculating means 

45 calculates an inter-frame moving amount of each lighting spot in an axial direction connecting the lighting pattern 

projecting means and the image pickup means while providing a positive or negative sign in accordance with a 
moving direction, and calculates an inter-frame moving amount of the lighting pattern by use of a moving amount 
of a pixel corresponding to the single or plural slit rays. 

so 20. The monitor as set forth in any one of Claims 1 through 1 6, wherein the lighting pattern projecting means projects 
a lighting pattern that has a single slit ray or a plurality of slit rays, 

and the moving amount calculating means calculates an inter-frame moving amount of each pixel corre- 
sponding to a slit ray in an axial direction connecting the lighting pattern projecting means and the image pickup 
means while providing a positive or negative sign in accordance with a moving direction, and calculates the total 

55 moving amount of each pixel corresponding to the slit ray as an inter-frame moving amount of the lighting pattern. 

21 . The monitor as set forth in any one of Claims 1 through 20, wherein the lighting pattern projecting means is disposed 
directly above a part close to one end of an edge of bedding facing a head or a foot of a person sleeping on the 
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and the image pickup means is disposed directly above a part close to an opposite end thereof. 

22. The monitor as set forth in any one of Claims 1 through 21 , comprising presence-in-bed detecting means for 
5 detecting the presence/absence of a sleeping person. 

23. The monitor as set forth in any one of Claims 1 through 22, comprising a pressure sensitive switch that can be 
placed under the sleeping person and can be used to detect the presence/absence of the sleeping person. 

10 
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